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Abstract 

In this paper, we discuss non-adaptive distributed compression of inter-node correlated real- valued messages. To 
do so, we discuss the performance of conventional packet forwarding via routing, in terms of the total network load 
versus the resulting quality of service (distortion level). As a better alternative for packet forwarding, we briefly 
describe our previously proposed one-step Quantized Network Coding (QNC), and make motivating arguments on 
its advantage when the appropriate marginal rates for distributed source coding are not available at the encoder 
source nodes. We also derive analytic guarantees on the resulting distortion of our one-step QNC scenario. Finally, 
we conclude the paper by providing a mathematical comparison between the total network loads of one-step QNC 
and conventional packet forwarding, showing a significant reduction in the case of one-step QNC. 
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1 Introduction 



In this paper, we study the incast of correlated real- valued messages (e.g. sensed data) in multi-hop networks. Usually, 
c/3 ' the messages in such networks need to be transmitted to a so called gateway node, which relays them to the next 
, level of the hierarchy. Delivery of these messages to the gateway node is based on the hopping nature of the network, 
in which the messages are forwarded, with the aid of a transport protocol. Specifically, in this paper, we review the 
information theoretic results on this so called data gathering (or incast) scenario. We will also motivate a discussion on 
^ ■ non-adaptive encoding of messages and describe our previously proposed transmission scheme, which tries to perform 
, as efficient as when the appropriate marginal rates for separate encoding were known. Our notion of non-adaptive 
■ encoding is with respect to the inter-node correlation of messages, and studies the cases, where the knowledge of 
, appropriate marginal rates are not available at the encoders' side. 

Initiated in ,T], network information flow analysis has helped develop more efficient transmission schemes, when 
, dealing with multiple communication agents. Explicitly, network coding has been suggested as an alternative for 
C^) ' routing-based packet forwarding, for which many theoretical and practical advantages have been listed. In the case of 
,— 1 \ lossless networks, with limited link capacities, cut-set upper bound has been shown to be the capacity region [1]. As an 
^» ■ extension to lossy networks, in [2], the capacity region is calculated for erasure networks, with side information at the 
decoder. Moreover, random linear network coding [5J, which can be implemented non-adaptively and computationally 
simple, is shown to be sufficient for multicast of messages in memoryless and lossless networks 0]. 

For the case of correlated sources (messages), performing distributed lossy or lossless source coding [5] has shown 
to be sufficient, even if adopted along routing-based packet forwarding [3 [3 [8]. However, advantages of network 
coding have drawn attention, especially in the case of lossy networks [S] • As a new advantage of network coding, in 
cases where the knowledge of appropriate information rates for performing distributed source coding is not available 
at the encoders' side, random linear network coding seems to be helpful. Specifically, this has been studied in [TUllll| . 
where we proposed Quantized Network Coding (QNC) and discussed practical feasibility of distributed source coding 
by semi-random linear network coding. 

In this paper, we study non-adaptive distributed compression of correlated messages (sources), by using network 
coding. In section [21 we describe the network and define the incast scenario for sparse messages. Then, in section [5J 
we briefly describe quantization and packet forwarding method of transmission and analyze its performance, in terms 
of the total network load versus the achieved distortion level (quality of service). In section [5J we describe a simplified 
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version of QNC, called one-step QNC [T2], where QNC and packet forwarding are combined. This simplification helps 
us to provide analytic results on the sufficient number of quantized network coded packets to ensure a given allowable 
distortion, as presented in section [5j Finally, in section [6l we present our conclusion remarks and discussions. 



2 Network Model and Notation 

To model the network, we use a directed graph, Q = (V, £), where V = {1, • • • , n}, and £ = {1, • • • , \£\}, are the sets 
of nodes and edges, respectively. The nodes denote the sensing and communication devices, and the edges represent 
communication links between pairs of nodes. Specifically, each edge, e, can maintain a lossless communication from 
its tail node, tail(e), to its head node, head(e), at a maximum rate of Co bits per channel use. Because of the lossless 
nature of the links, the input and output contents of edges are the same and denoted by Y e (t), at time index t. This 
time index represents transmission of a packet of length L over edges. We also assume that the initial rest condition 
holds in the network, implying: 

F e (l) = 0, Vee£. (f) 
Further, the sets of incoming and outgoing edges of node v, are also defined as follows: 

In(v) = {e £ £ : head(e) = v}, (2) 
Out(v) = {e £ £ : tail(e) = v}. (3) 

The edges are uniformly distributed between pairs of nodes, forming a Erdos-Renyi random graph j!3j . Explicitly, 
for each edge e £ £ , and for all v, v' £ V, where v ^ v' , we have: 

p(tail(e) — v, head(e) = v'\ = — r. (4) 

V / n(n — 1) 

Associated to each node v, there is a random variable X v £ R, which is considered as the corresponding message 
of that node. These random messages, X_ = [X v : v £ V], are correlated and their correlation is modeled by using 
sparsity. Specifically, we assume that there is an orthonormal transform matrix, 4> n xn, for which S — <fi T 2L is fc-sparse: 
= k. Moreover, we assume that the messages, take their values randomly and uniformly between -g max and 

+q max ■ 

\x v \ < g max , v« e V. (5) 

The messages, X v , are ready for transmission at t = 1. In this paper, the lower case letters correspond to the 
realizations of random variables, denoted by upper case letters. 

In our incast scenario, referred to as data gathering, we need to transmit messages, X v J s, to a single gateway 
(decoder) node, denoted by vq £ V and recover them with a maximum distortion of Do, such that: 

E[|X„ -X v \}< D , VueV. (6) 

In ©, X v is the recovered version of X v , at the decoder node. The product of the required number of packets to 
ensure the above distortion constraint and the packet length is called the total network load, in this paper. This total 
network load can be used to reflect the required number of transmissions and can be used as a measure of efficiency 
for different transmission methods. 

Performing noiseless |14) or noisy |15j distributed source coding is the usual solution to take care of inter-node 
redundancy of messages. However, in the described cases, where the knowledge of appropriate marginal rates for 
performing separate encoding is not available at the encoder side, only a non-adaptive scheme, which does not rely on 
the prior of messages, can be adopted. In this paper, we study such scenario and discuss on the feasibility of employing 
efficient non-adaptive transmission schemes for sparse messages. 



3 Quantization and Packet Forwarding 

In this section, we consider Packet Forwarding (PF) as a non-adaptive transmission method for the messages. As it 
was discussed earlier, distributed source coding can not be done, as the appropriate marginal rates are not known at 
the sensor (encoder) nodes. The messages are going to be forwarded to the decoder node, according to the calculated 
routes. 
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Limited capacity of the links requires us to quantize the messages before transmission over the links. This has 
motivated the use of phrase Quantization and Packet Forwarding (QPF), for this method. Specifically, we quantize the 
messages and send the quantized version, Q(X V ), to the decoder node. Since the messages are uniformly distributed, 
a uniform quantizer minimizes the associated quantization noise power, for which: 

|^-Q(^)|<A Q = ^. (7) 
Moreover, for the uniform quantizer, we have: 

n\X v -Q(X v )\] = ££. (8) 

This is summarized in the following corollary. 

Corollary 3.1 For the described QPF scenario with real-valued uniform messages, the distortion level of Dq, 

E[\X V -X V \] <D Q , Vve V, (9) 
is achieved if and only if the adopted packet length, L, to transmit (n — 1) quantized messages is such that: 

Lc-Llog 2 fe, (10) 

resulting a total network load of 

Mn-l) = ^log 2 fe. ■ (11) 

It can be understood from this corollary that the total network load is in the order of the number of nodes, n. In 
section [SJ we will see that by using our previously proposed one-step QNC, one requires a smaller load to ensure the 
same distortion level. 



4 One-Step Quantized Network Coding 

Our formulation of QNC was originally presented in [TU1 HH] , where the sparse recovery was the only criterion used 
to recover the messages at the decoder node. In IT], we investigated the possibility of implementing optimal mean 
square-error decoder, when the prior information of messages is also known, in addition to their sparsity. One-step 
QNC (originally mentioned in [T2]) is a special case of QNC, in which we perform linear network coding only at the 
first time instance and then simply forward the quantized network coded packets to the decoder node. In the following, 
we explicitly describe our one-step QNC scenario and its explicit formulation. 

To initiate one-step QNC, each node transmits the quantized messages, Q(X„)'s, to its neighboring nodes. As a 
result of initial rest condition, at t = 2, we have: 

F e (2) = Q(X tail(e) ) (12) 
= X tall(e) +N e (2), (13) 

where N e (2) is the quantization noise. These quantized messages are used in each node v to calculate a random linear 
combination, P v : 

Pv = /V y e'(2) + a v X v , to e V, (14) 

where the local network coding coefficients, /8 V)e ''s and a v 's, are uniformly and randomly selected from {— k, +k}, 
k > 0. The appropriate value of the constant k was shown in [12] to be equal to: 

/ 2n 2 , . 

K = vWr (15) 

Moreover, to avoid over flow, we ensure that the normalization condition of Eq. 3 in |10] holds, implying: 

\0v,e>\ + \Ov\<l, VweV. (16) 
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Next, each node randomly and independently decides to forward each P v to the decoder node or not, according to 
a fixed binary probability model. Representing the i'th received packet at the decoder node by {Z_ tot (t)}i, we have: 

{Z M (t)h = Pv+N e (3), w — >i, (17) 

where v — > i means that P v is forwarded to decoder and corresponds to the i'th received packet. The outgoing edge 
e, e s Out{v), is also the one on which Q(P„) is sent out and therefore N e {2>) is the corresponding quantization noise. 
The total number of received packets at the decoder node, by time instance t, is represented by m. 
Linearity of this scenario lets us reformulate Z_ tot (t) as: 

Z tot (t)=V tot (t)-X + N cSttot (t), (18) 

where ^tot(t) and N cS tot (t) are the total measurement matrix and the total effective noise vector, respectively: 

, v' — > i, v v', 

, w '^i,u' = u, (19) 
, otherwise 

{iVeff,tot(*)}i=^e(3)+ E Pv,e' N e ,(2), V ^ I. (20) 
e' £/n(v) 

In (fT9")l . w A u' denotes that there is an edge e from u to v'. 

Motivated by the theory of compressed sensing and sparse recovery [T71 [IB], we claimed that one can recover 
messages from smaller number of received packets, {Z tot (£)}j's, than the number of messages, i.e. m < n 12j. The 
decoder design and implementation for achieving near-optimal (mean squared error) performance is discussed in [11) . 
In the next section, we investigate the performance of one-step QNC, by analyzing its total network load for a given 
allowable distortion level, Dq. 



{*tot(£)kt 



Pv' ,e 





5 Distortion Analysis for One-Step Quantized Network Coding 

The precise distortion analysis for QNC scenario can not be done easily because of its reliance on sparse recovery 
algorithms, for which a comprehensive statistical distortion analysis is still not available. However, we are still able to 
obtain some performance bounds on the delay-quality performance, thanks to the work on Bayesian compressed sensing 
[T9"j . In the following, we present a series of theoretical results, which characterizes the delay-distortion performance 
of one-step QNC scenario for non-adaptive encoding of sparse messages. 

Theorem 5.1 For the one-step QNC scenario, described in section^ where X_ is k-sparse in the transform domain 

finxn, With 

ffmax = max||(£ T , (21) 



then for any e, 7 > 0, using a number of packets, m, where 



(k 2 - 1) kq' 2 + A% 
m>48(l +l y- '—^ ^log(n), (22) 



one can decode Z_ tot (t) into X_, such that: 

P(\X V -X v \<e)>l- n~ 7 , Vw e V. (23) 

Proof See Appendix. 

Theorem 5.2 For the described one-step QNC scenario, if the total network load, m ■ L, satisfies the condition of 
Eq. \25[ then we have: 

E[\X V -X V \] <D Q , \/v e V. (24) 
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(k 2 - 1 1 kn' 2 -4- n 2 92-21/ C 

m-L > min 48(1 + 7)^ '- q ^J q ^ V ■ log(n) (25) 

subject to: e(l — n~ 7 ) + 2<7, nax n~ 7 < Dq. 

{k 2 -Wkn' 2 +n 2 9 2 -2Co 

m-L > 96 [K ^"- + ^ log(n) (26) 

(D - 2g max /n) 2 



Proof From theorem 15. 11 we know that if for a choice of e, 7 > the condition of (|22|) is satisfied, then the expected 
value of \X V — X v \ is smaller than e, with probability 1 — nT 1 . We also know that X v 's are bounded between — g max 
and +g max . Therefore, if we limit (clip) the outcome of the decoder, X v , to be between — g max and +g max , we can 
ensure that \X V — X v \ < 2g max , for the portion of the times (n~ J ) that theorem 15.11 fails to provide any guarantee. 
Hence: 

V[\X V -X v \}< e(l - n-T) + 2g max n-T. (27) 

And, by constraining: 

e(l - n-T) + 2g max n-^ < D 0) (28) 

we can also ensure that E[|X„ — < Do holds. Finally, we perform a minimization to find the lowest possible total 
network load, subject to the constraint ((24]) . 



As it can be understood from (|25D . increasing the packet length, L, helps to decrease the required number of 
packets, m. On the other hand, it may increase the total network load. Theorem 15.21 formalizes this trade-off and 
characterizes the appropriate choice of packet length. 

By choosing 7 = 1, we obtain: 

e <" jQ " 2fa (29) 
n — 1 

And since 

„ 2g max nDo - 2q max nD - 2<j max 

L>o — < , (oil) 

n n n — 1 

by choosing e = Do — 2q ma _ x /n and L = 1, theorem 15.21 simplifies to the following corollary. 

Corollary 5.3 For the one-step QNC scenario, if the total network load, m-L, satisfies the condition of Eg. Wb\ then 
the desired distortion level of Do can be ensured: 

E[\X V - X v \] < D Q , Vv S V. ■ (31) 

Although the preceding corollary loosen our former formulation on the conditions to ensure the distortion constraint, 
it provides us with a better understanding. Explicitly, when the number of network nodes, n, is large, the total network 
load in one-step QNC scenario, m • L, (Eq. 126]) is in the order of log(n). As discussed in section [3l the total network 
load in QPF scenario, {n — 1) • L, has order of n, for the same distortion level. This clarifies a significant reduction in 
the total network load in one-step QNC scenario, for large networks. 



6 Conclusion 

The possibility of non-adaptive distributed compression of correlated sources has been investigated in this paper. This 
was done by discussing the performance of our previously proposed one-step QNC scenario, in which random linear 
network coding meets the theory of compressed sensing and sparse recovery. Specifically, we derived a sufficient (not 
necessary) condition on the network load to satisfy a desired distortion level. Our mathematical derivations show a 
significant decrease in the total load of the network, compared to the case of conventional QPF. The resulting decrease 
in the total network load shows a potential decrease in the overall transmission power in the network. One may also 
weakly interpret this decrease as a decrease on the delivery delay, required to achieve the desired distortion level. 
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Appendix: Proof of theorem 15.11 

It is shown in the proof of theorem 3.1 in [T2J that \l/tot(i) is such that for Vi, v, 1 < i < m, 1 < v < n, {&tot(t)}i,v 's 
are independent, and we have (Eq. 20 in [12]): 

E[{*tot(*)}<«] = 0, 
E[{* tot (i)} 2 J = 1, 

E[{*tot(*)}?„] = ^- (32) 

Define positive integers mi and m,2, which we will determine, and set m = tti\tti2 • Partition the m x n matrix 
^tot(t) into mi matrices {^i, . . . ,^ m2 }, each of size m± x n. Now, consider a realization of X, called x, for which 
we define {z_ x — ■^= , &ix + n 1 , . . . ,z m2 = ■^=^m 2 2L + ZL m2 }- Moreover, {n ; }i's are the corresponding elements of the 
total effective noise vector, for which (I2D]1 implies: {n;}^ < Aq. 

Furthermore, we define u/s, where 1 < j < n, to be the canonical vectors in R™. For them, we define {z_[ ■ = 

Now, we define random variables r% t j, . . . , r m2j -: 

mi mi 1 



i=l i=l 1 



where: 



Moreover, nj, . . . ,r m2 j are independent because of the independence of {'J/ku's. 
By using a similar reasoning as in |20j . we have: 

n 

Efwij,.] = E[^{*;}^ x v {UjU 

v=l 

n n 

v—1 v'—v+l 
n 

n 

n n 

e^j = e[(^{^,k^) 2 • (£{^k„{%k) 2 ] 

U — 1 1 

n 

+fe} 2 E[(£{v^k„{ % k) 2 ] 

v=l 

n n 

+2{n ; }, EK^^k^X^*,}^},,) 5 



U — 1 u = l 

2(£ T -w i ) 2 + ||s|| 2 2 ||% ||1 



,2 



where in the last equality we used Lemma 1 in [2D], and also the fact that E[{\&/}| J = 0. Recalling that u/ s are 
canonical vectors with unit ^2-norm, we have: 

Var[«J Wli ] = ||x|| 2 2 + (K 2 -2)x 2 + fe} 2 . 







Now, for 77 j, we have: 



mi 

E Kj] = E E[w M ] = x T ■ u 



. mi 



TO, 

1 8=1 



mi 

Thus, the Chebyshev inequality, 



^-[\k\\l + (^-2)x 2 j + {n l }] 



„/, t , n Var(r; .,) 
Pdr^-x^l^e) < 

and the fact that < Aq, imply: 

Pdry-^-l^e) < -J_(|| 2I ||, 2 2 + ( K 2 -2)^+A^) 

- ^(^ 2 - 1 ) m i x i^ii +A Q)- 

By picking 

(k 2 — 1) maxr ||a;|| 2 + A?, 
to! > 4^ 1 — , (33) 

we have: 

P(ki-Si|>e) < J. (34) 

We define the decoded value for Xj, to be the median of the independent random variables rij, . . . , r m2 j. 
Further, we define the indicator random variable to be equal to one if |ry — Xj\ > e and zero otherwise. Now, if 
Y]l—i is more than the median (decoded value) will not be inside the interval: \£j — Xj\ > e. Moreover, we 
have: E[J^Li < By using the Chernoff inequality 

m 2 m 2 

p(E fw > a + *) E E 6 ^ e-^-ra & -]/> (35 ) 



;=i {=1 



where < (5 < 1 1211. we obtain: 



pEfijXi+^TX^" (36) 



1=1 



Taking the limit of when 6—^1 , 

m 2 

p(E^>?)< e ~ m2/12 - ( 37 ) 
;=i 

Finally, by using the union bound, we have: 

P(3j : \ Xj - Xj\ > ej < n • e~ m2/12 . (38) 

Therefore, by choosing 

m 2 >12(l + 7)Iog(n), (39) 
and combining with (1331) to find m = mi ■ 7712, we have proved the theorem. 
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